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Abstract 

We  review  the  asymptotic  theory  for  standard  errors  in  classical 
ordinary  least  squares  (OLS)  inverse  or  parameter  estimation  prob¬ 
lems  involving  general  nonlinear  dynamical  systems  where  sensitivity 
matrices  can  be  used  to  compute  the  asymptotic  covariance  matrices. 
We  discuss  possible  pitfalls  in  computing  standard  errors  in  regions  of 
low  parameter  sensitivity  and/or  near  a  steady  state  solution  of  the 
underlying  dynamical  system. 
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1  Introduction 


There  is  a  growing  interest  in  measures  of  uncertainty  in  many  areas  of 
mathematical  modeling  as  more  sophisticated  models  are  employed  in  biol¬ 
ogy  [4,  3,  8],  chemistry,  sociology  [7],  etc.,  as  well  as  in  the  usual  physical 
and  engineering  sciences.  In  particular,  there  has  been  recent  renewed  ef¬ 
forts  by  applied  mathematicians  in  the  area  of  inverse  or  parameter  estima¬ 
tion  problems  to  attach  measures  of  reliability  to  estimated  quantities  using 
experiment  data  sets.  Statisticians  have  for  some  time  [9,  11,  15,  16,  19] 
recognized  the  importance  of  such  problems  and  have  developed  theory  and 
methodology  to  treat  these.  In  particular,  a  rather  extensive  area  of  matu¬ 
rity  is  the  computation  of  standard  errors  (SE)  and  confidence  intervals  (Cl) 
for  ordinary  least  squares  (OLS)  problems  using  what  engineers  and  applied 
mathematicians  call  sensitivity  matrices  [1,  10,  12,  13,  14,  18].  These  can  be 
computed  or  approximated  in  a  number  of  ways  and  for  quite  sophisticated 
unknown  “parameters”  including  functions  and  even  probability  measures 
[2,  3,  5,  6].  A  most  frequently  used  framework  for  the  computation  of  SE 
is  the  asymptotic  theory  of  estimators  which,  roughly  speaking,  guarantees 
under  certain  conditions  that  as  one  uses  more  and  more  data  in  the  inverse 
problems,  that  one  can  approximate  the  estimator  or  sampling  distribution 
by  a  Gaussian  with  computable  mean  and  covariance.  As  with  most  statisti¬ 
cal  theories,  this  asymptotic  theory  holds  under  quite  specific  hypotheses  on 
the  statistical  model  assumed  for  the  observation  error  (which  may  include 
both  model  and  measurement  error)  as  well  as  assumptions  on  regularity  of 
the  underlying  problem  and  the  method  in  which  the  increasing  amount  of 
data  is  collected.  In  practice,  most  investigators  are  concerned  with  the  form 
of  the  error  (e.g.,  independent  identically  distributed  with  constant  variance 
or  constant  coefficient  of  variation)  but  do  not  give  much  concern  to  either 
the  regularity  hypotheses  or  the  method  of  data  collection.  This  can  some¬ 
times  lead  to  surprising  and  seemingly  perplexing  results  for  unwary  users. 
The  purpose  of  this  note  is  to  recall  these  results,  discuss  some  pitfalls,  and 
illustrate  with  an  example  that  shares  qualitative  features  with  many  systems 
encountered  in  wide-spread  scientific  applications.  We  focus  specifically  on 
the  phenomenon  that  increased  sampling  data  often  does  not  result  in  im¬ 
provements  in  the  estimates  or  in  their  associated  statistical  reliability  as 
embodied  in  their  SE  or  Cl.  For  example,  when  additional  data  is  sampled 
from  a  region  near  an  equilibrium  or  steady  state,  the  estimated  values  of 
the  problem  generally  do  not  improve.  In  fact,  the  confidence  intervals  for 
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estimated  values  often  increase  in  size  as  more  data  is  taken.  To  discuss  the 
causes  for  this,  we  will  illustrate  ideas  with  the  logistic  growth  population 
model. 


2  Summary  of  asymptotic  theory  for  errors 

We  first  give  a  general  summary  of  the  asymptotic  theory  for  standard  errors. 
We  assume  that  n  scalar  longitudinal  observations  (the  extension  to  vectors 
is  completely  straight-forward)  are  represented  by  the  statistical  model 

Yj  =  fj(P o)  +  <b,  .7  =  1,2,..., n,  (1) 

where  fj(Po)  is  the  model  for  the  observations  in  terms  of  the  state  variables 
and  p0  e  is  a  “set”  of  theoretical  “true”  parameter  values  (assumed 
to  exist  in  a  standard  statistical  approach).  For  example,  if  one  is  given 
a  differential  equation  system  x  =  T(f,  x(t),  (3),  sampled  at  times  tj,j  = 
1,2,...,  n,  then  /_,■(/?)  =  x{tj,  (d). 

We  assume  for  our  statistical  model  of  the  observation  or  measurement 
process  (1)  that  the  errors  €j,  j  =  1,2, . . .  ,n,  are  independent  identically 
distributed  ( i.i.d .)  random  variables  with  mean  E[ej]  =  0  and  constant 
variance  var[ej]  =  <7q,  where  of  course  a q  is  unknown.  We  then  have  that  the 
observations  Yj  are  i.i.d.  with  mean  E[Yj]  =  fj(/3o)  and  variance  var[Yj\  = 

We  consider  estimation  of  parameters  using  an  ordinary  least  squares 
(OLS)  approach.  Thus  we  seek  to  use  data  {yj}  for  the  observation  process 
{Yj}  with  the  model  to  seek  a  value  f3n  that  minimizes 

n 

urn  =  E  i  w)  -  »i2-  (2) 

3= 1 

Since  Yj  is  a  random  variable,  we  have  that  the  estimator  (3qLS  is  also  a  ran¬ 
dom  variable  with  a  distribution  called  the  sampling  distribution.  Knowledge 
of  this  sampling  distribution  provides  uncertainty  information  (e.g.,  standard 
errors)  for  the  numerical  values  of  (3n  obtained  using  a  specific  data  set  {yj} 
(i.e.,  a  realization  of  {Yj})  when  minimizing  Jn(/3). 

Under  reasonable  assumptions  on  smoothness  and  regularity  (the  smooth¬ 
ness  requirements  for  model  solutions  are  readily  verified  using  continuous 
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dependence  results  for  differential  equations  in  most  examples;  the  regularity 
requirements  include,  among  others,  conditions  on  how  the  observations  are 
taken  (more  on  this  later)  as  sample  size  increases,  i.e.,  n  — »  oo),  the  stan¬ 
dard  nonlinear  regression  approximation  theory  ([11],  [15],  [16],  and  Chap¬ 
ter  12  of  [19])  for  asymptotic  (as  n  — >  oo)  distributions  can  be  invoked. 
This  theory  yields  that  the  sampling  distribution  /3n(Y)  for  the  estimate  f3n , 
where  Y  =  {Yj}7j=1,  is  approximately  a  p- multivariate  Gaussian  with  mean 
E[(3n{Y)]  and  covariance  matrix  cov[/3n(Y)\  kS0  =  ffi)[xT(A))x(A))]_1-  Here 
x(/3)  =  Fpiji)  is  the  n  x  p  sensitivity  matrix  with  elements 

Xjk(P)  =  and  FM  =  (MP)>  •  •  •  > fnM)T- 

That  is,  for  n  large,  the  sampling  distribution  approximately  satisfies 

Pols(Y)  ~  KWo,°Z\xrWo)x(Jh)]-1)  :=y,(A,Eo).  (3) 

There  are  typically  several  ways  to  compute  the  matrix  Fg.  First,  the 
elements  of  the  matrix  y  =  (xjk)  can  always  be  estimated  using  the  forward 
difference 

M  =  9fM  „  fjjP  +  hkhJM, 

dfik  hk 

where  hk  is  an  p- vector  with  nonzero  entry  in  only  the  kth  component.  Al¬ 
ternatively,  if  the  fj(/3 )  correspond  to  longitudinal  evaluations  x(tj,/3)  of 
solutions  x  G  Mn  to  a  parameterized  n*-vector  differential  equation  system 
x  =  r (t,x(t),/3),  then  one  can  use  the  n*  x  p  matrix  sensitivity  equations 
(see  [6]  and  the  references  therein) 

d  f  dx\  dT  dx  <9T 
dt  \d/3 )  dx  d/3  +  dj3 

to  obtain 

dfj(P)  =  dx(tj,P) 

dpk  dfk 

Finally,  in  some  cases  the  function  fj(/3)  may  be  sufficiently  simple  so  as  to 
allow  one  to  derive  analytical  expressions  for  the  components  of  Fg.  This  is 
the  case  for  the  problems  addressed  in  this  paper  and  hence  for  our  efforts 
we  chose  this  latter  approach  in  the  examples  below. 
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Since  /30,  cr0  are  not  known,  we  must  approximate  them  in 


So  =  °l[xT  %)x%)\ 

For  this  we  follow  standard  practice  and  use  the  approximation 

£„  «  E(/3”)  =  #2[xTd")x(/3“)]_1  (4) 

where  /3n  is  the  parameter  estimate  obtained,  and  the  approximation  b2  to 
cr(j  is  given  by 

1  n 

(s) 

7  j=i 

Standard  errors  to  be  used  in  conhdence  interval  calculations  are  thus  given 

by  SEk0n )  =  \jEkk0n),  k  =  1,  2, . . .  ,p  (see  [9]). 

Finally,  in  order  to  compute  the  conhdence  intervals  (at  the  100(1  —  a)% 
level)  for  the  estimated  parameters  in  our  example,  we  define  the  conhdence 
level  parameters  associated  with  the  estimated  parameters  so  that 

P{fa  -  t^SEkCn  <  ft  <  pnk  +  t^a/2SEkCn}  =  1  -  a,  (6) 

where  a  G  [0, 1],  and  ti_a/2  G  M+.  For  a  given  a  value  (small,  say  a  =  .05 
for  95%  conhdence  intervals),  the  critical  value  ti_a/2  is  computed  from  the 
Student’s  t  distribution  tn~p  with  n  —  p  degrees  of  freedom.  The  value  of 
ti-a/2  is  determined  by  P{T  >  ti_a/2}  =  a/2  where  T  ~  tn~p. 

We  note  that  when  fj(/3)  =  x(tj,/3)  as  mentioned  above  when  one  is 
taking  longitudinal  samples  corresponding  to  solutions  of  a  dynamical  sys¬ 
tem,  the  n  x  p  sensitivity  matrix  depends  explicitly  on  where  in  time  the 
observations  are  taken.  Note  also  that  the  sensitivity  matrix 


X(P)  =  FM 


(d_m\ 

V  df3k  ) 


depends  on  the  number  n  and  nature  (e,g.,how  taken)  of  the  sampling  times 
{tj}.  Moreover,  it  is  the  matrix  [yTy]_1  in  (4)  and  the  parameter  a2  in 
(5)  that  ultimately  determine  the  SE  and  CF  At  hrst  investigation  of  (5), 
it  appears  that  an  increased  number  n  of  samples  will  drive  a2  (and  hence 
the  SE)  to  zero  as  long  as  this  is  done  in  a  way  to  maintain  a  bound  on 
the  residual  sum  of  squares  in  (5).  But  then  we  observe  that  the  condition 
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number  of  the  matrix  is  also  very  important  in  these  considerations 
and  increasing  the  sampling  could  possibly  adversely  affect  the  inversion  of 
XTX-  In  this  regard,  we  note  that  among  the  important  hypotheses  in  the 
asymptotic  statistical  theory  (see  p.  571  of  [19])  are  that 

^XT((3)x((3)  -*■  X{P)  as  n  — >  oo 

with  X(/3q)  nonsingular.  It  is  this  condition  that  is  rather  easily  violated 
in  practice  when  one  is  dealing  with  data  from  differential  equation  systems 
near  an  equilibrium  or  steady  state.  To  illustrate  this  as  well  as  related  issues 
regarding  sensitivity  that  are  often  ignored  in  statistical  discussions,  we  turn 
to  a  simple  example,  the  popular  logistic  growth  or  Verhulst-Pearl  equation 
[17],  where  analytic  expressions  are  readily  found  and  used  to  discuss  the 
ideas. 


3  Logistic  growth  population  models 

We  consider  the  dynamics 


dx 

dt 


rx 


l  -  — 
K 


where  K  is  the  carrying  capacity  and  r  is  the  intrinsic  growth  rate.  This  is 
the  Verhulst-Pearl  logistic  equation  [17].  The  exact  solution  is  readily  solved 


Figure  1:  Solution  to  ODE 
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for  and  is  given  by 


K 


x(t) 


This  function  has  a  flex  point  (  Cf  =  0  I  at  x 


dt 2 


We  will  examine  the  equivalent  systems 


K 

2  • 


x  =  ax  —  bx 2, 


(7) 


which  have  solutions 


a:(t) 


a/b 

a 

b  +  /ce_at 


(8) 


with  k  =  —  b.  The  solutions  x(f)  have  an  asymptote  as  f  — »  oo  at  x°°  = 

|  =  K .  Note  that  a  =  r  and  hence  j-.  —  b.  When  xq  >  K,  the  solutions 
x{t)  are  monotone  decreasing  functions  with  an  asymptote  at  K .  Similar 
behavior  (monotone  increasing)  is  observed  for  solutions  with  Xq  <  K. 

Standard  errors  will  be  examined  for  this  example  using  both  analyti¬ 
cal  calculations  as  well  as  numerical  computations.  The  analytical  deriva¬ 
tions  will  show  that  if  observations  are  taken  in  specific  regions  pertaining 
to  the  solution  curve,  yTy  could  approach  a  singular  matrix,  resulting  in  the 
standard  errors  computed  with  the  asymptotic  theory  becoming  unbounded. 
We  expect  that  the  numerical  computations  performed  using  MATLAB  will 
behave  similar  to  the  analytical  results  and  therefore  expect  that  the  cal¬ 
culations  will  go  badly  if  observations  are  not  correctly  chosen.  This  will 
reinforce  the  observation  that  in  practical  problems,  sensitivity  equations 
and  standard  errors  can  be  used  to  inform  experimental  design  (e.g.,  so  that 
observations  are  taken  appropriately  to  avoid  such  difficulties). 


4  Analytical  considerations 

As  stated  earlier,  we  consider  an  ordinary  least  squares  problem  for  the  pa¬ 
rameters  [3  =  (a,  b,  Xq)  with  the  approximate  covariance  matrix  given  by 

£  =  Ptfx)-1 
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and  the  corresponding  estimate  of  the  standard  error 


SEj  =  \Ja2{xTx)jj,  j  =  l...n.  (9) 

Recall  that 

£  =  °2(XTXY1 

p  j=1 

=  —  (RSS)(xtx)-1 

n  —  p 

where  RSS  denotes  the  residual  sum  of  squares.  In  this  example  we  have 


dxT 

dj3 


I  dx(t\) 
da 

dx(ti) 

db 

dx(ti) 
dx  o 


dx(t.n)  \ 
da 

dx(tn) 

db 

dx(tn) 
dx0  / 


We  will  examine  varying  behavior  in  the  model  and  the  resulting  sta¬ 
tistical  analysis  depending  on  the  region  from  which  tj  is  sampled.  To  this 
end,  let  R0  be  the  region  where  t  G  [0,  Ti],  R\  be  the  region  corresponding 
to  t  G  [ti ,  T2] ,  and  R2  where  t  G  [r2,cx))  as  depicted  in  Figure  2.  We  expect 
differences  in  the  ability  to  estimate  parameters  depending  on  the  region  in 
which  the  solution  is  observed  (sampled). 


Figure  2:  Partition  of  solution  curve  into  distinct  regions. 


We  will  examine  behavior  and  anticipate  Ending  three  distinct  situations. 


1.  When  data  is  sampled  from  region  R0,  the  solution  is  insensitive  to 
the  parameters  a  and  b.  If  we  attempt  to  estimate  a  or  b  from  this 
data,  we  expect  to  obtain  large  standard  errors  as  xTX  approaches  a 
singular  matrix  as  n  — >  oo.  Thus,  the  parameters  a  and  b ,  as  well  as  the 
carrying  capacity,  K  =  a/b ,  cannot  be  estimated  using  data  sampled 
from  R0  alone. 

2.  When  sampling  in  Ri  we  suspect  that  estimating  (3  with  decent  stan¬ 
dard  errors  is  possible,  and  using  more  data  (up  to  a  point)  will  improve 
reliability  (i.e.,  the  corresponding  SE). 

3.  When  data  is  sampled  from  R2,  the  solution  is  insensitive  to  Xq.  If  we 
try  to  estimate  x0  from  this  data,  we  expect  to  obtain  large  standard 
errors  because  should  again  become  ill-conditioned  (approximately 
singular)  as  n  — >  oo.  We  cannot  estimate  a  and  b  independently,  how¬ 
ever  we  should  be  able  to  estimate  the  carrying  capacity  K  =  a/b  since 
x(t)  — >  K  =  x°°  as  t  — >  oo. 


In  order  to  consider  the  problem  for  data  in  different  regions,  we  explicitly 
solve  the  equation 

x  =  ax  —  bx2  (10) 

for  x ,  and  then  examine  the  partial  derivatives  |^,  and  ////.  Rewriting 
the  equation  as 

dx 

x{a  -  bx)  = 

we  obtain  (by  integrating  and  using  partial  fractions)  the  solution 

a 

x  = 


b  +  ke  at 

Thus,  we  readily  see  that  x(t)  — >  K  —  |  as  t  — >  oo.  Also  note  that  x(t)  — >  ^ 
as  t  — >  0.  We  will  therefore  denote  x(0)  =  xq  = 

We  easily  compute  the  partial  derivatives 


dx 

b  +  ( atk  —  b)e 

da 

( b  +  ke~at)2 

dx 

— a(l  -  e~at) 

8b 

( b  +  ke~at)2 

dx 

a2e~at 

dx  o 

xl(b  +  ke~at)2  ’ 
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The  matrix  x  is  thus  composed  of  these  partial  derivatives  evaluated  at  points 
t  t\ , . . . ,  tn.  i.e. , 


dx{t\) 

dx(ti) 

dx{t\) 

da 

db 

dxo 

dx(tn) 

dx(tn) 

dx(tn) 

da 

db 

dx0 

For  use  in  approximation  E  of  the  covariance  matrix  we  thus  obtain  the 
matrix 


xTx  = 


( dx(tj)\  dx(tj)  dx(tj)  dx(tj)  dx(tj) 

y  da  J  da  Ob  da  dxo 

dx(tj)  dx(tj)  f  dx(tj)\^  dx(tj)  dx(tj) 

db  da  y  db  J  db  dxo 

dx(tj)  dx(tj)  dx(tj)  dx{tj)  ( dx(tj)\  ^ 

dxo  da  dxo  db  y  dxo  ) 


We  next  consider  the  different  regions  in  which  we  analyze  E.  We  address 
the  asymptotic  regions  R0  and  R2  first. 

If  we  sample  data  from  Rq,  where  tj  <  T\  for  j  —  1 ...  n,  we  have 


dx(tj) 

da 


as  a  result  of 


dxttj)  ^ 
db 


lira  =  0, 

t—>o  da 

lira  =  0. 

t—  o  db 

lira  d4*bi  =  1. 

t-> o  dx o 


dxtti)  w 

dx  o 


Also  note  that 


AA  dx(tj )  dx(tj 

^  dxn  dxr> 
r  =• 


J21  =  n 

3= 1 


for  tj  <  T\  in  Rq.  Hence,  in  this  region,  we  can  make  the  approximation 


/0  0  0 

xTx  ~ooo 

\0  0  n 
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In  R0  we  expect  not  to  be  able  to  accurately  determine  a  or  b,  however 
we  should  be  able  to  estimate  xq.  Moreover,  we  must  expect  difficulties  when 
computing  the  standard  errors  for  estimates  of  a  and  b.  As  additional  data 
points  are  sampled,  SEa  and  SEb  will  increase  because  (xTy)_1  approaches 
a  matrix  with  unbounded  values  in  the  first  two  diagonal  elements. 

Next  we  consider  the  region  R2l  where  tj  >  r2  for  j  —  1 . .  .n,  and  note 
that 


lim  dEE 

t-*  oo  da 
dx(tj) 


1 

V 


lim 

t—*  oo  db 

lim  =  0. 

t—>oc  OX  n 


a 


This  implies  that  in  R2 

dx(tj)  _ 


and  hence, 


da 


1 

V 


dx{tj 

db 


a 


dx(tj 

dxn 


0, 


T 

X  X 


E 

3= 1 


/ dx(tj)  dx(tj)  dx(tj)  dx(tj) 

'  da  da  da  db 


dx(tj)  dx(tj)  dx(tj)  dx(tj) 


db  da 


db  db 

o 


o\ 

0 

0 ) 


=  n 


(  i  yfi 


_ a_  ar_  n 

63  64  U 


\  0  0  0/ 


In  this  case  the  second  column  of  xTX  is  a  scalar  multiple,  —  | ,  of  column  one, 
and  thus  it  is  not  possible  to  estimate  a  and  b  independently.  Similar  to  the 
standard  errors  corresponding  to  region  Rq,  SEa  and  SEb  will  increase  in  size 
with  the  additional  sampling  of  data  points.  Also,  in  R2  there  is  difficulty 
estimating  Xq  with  any  accuracy.  The  ill-posedness  (ill  conditioning)  of  xTX 
can  be  expected  when  computing  the  standard  error  of  rc0,  and  SEX0  will 
also  increase  as  more  data  points  are  sampled. 
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In  region  R\ ,  where  T\  <  tj  <  t2  for  j  —  1, . . .  n,  we  note  that  the  partial 
derivative  estimates  differ  greatly  from  the  estimates  in  regions  Rq  and  R2, 
and  in  general  the  matrix  xTX  will  be  well-conditioned.  When  Ri  is  included 
in  the  sampling  region,  we  should  be  able  to  recover  decent  estimates  for  a, 
b,  and  Xq  along  with  reasonable  corresponding  SE. 

5  Implementation 

We  first  considered  inverse  problems  with  exact  (no-noise)  simulated  data. 
MATLAB  was  used  to  compute  numerical  solutions  to  equation  (7)  for  use 
as  the  model  {fj(/3)}  =  {f(tj,/3)}  in  (2).  We  create  a  simulated  data  set, 
(%}i=i ,  using  the  solution  given  by  (8)  with  a  specific  f3,  denoted  (3q.  That 
is,  we  evaluate  the  analytic  solution  (8)  at  tj  to  obtain  data  yj.  We  restrict 
the  values  of  tj  to  the  region  from  which  data  is  sampled,  and  provide  an 
initial  estimate  (3°  for  the  MATLAB  ODE  solver. 

The  cost  function  uses  odel5s  to  approximate  the  solution  and  returns 

n 

UP)  =  Y, \f(h,P)-Vi\2 

3= 1 

where  )  =  x(t;  a,b,x0)  is  the  numerical  approximation  to  the  solution. 
We  use  the  MATLAB  function  fminsearch  to  optimize  over  /3  in  order  to 
obtain  the  minimized  cost  Jn((3n).  Here  (3n  represents  the  optimized  value  of 
/3  using  n  data  points. 

6  Results 

The  following  are  the  results  from  three  sets  of  simulated  data:  a  relatively 
flat  curve  with  (3q  =  (0.5, 0.1, 0.1),  a  moderately  sloped  curve  with  /30  = 
(0.7,0.04,0.1),  and  a  steep  curve  with  /30  =  (0.8,0.01,0.1).  For  all  three 
curves,  define  R0  to  be  the  region  where  t  G  [0,2],  Ri  is  the  region  where 
t  E  [2, 12],  and  R2  is  where  t  E  [12, 16].  Within  each  region,  we  record  four 
different  initial  guesses  for  the  vector  /3°  =  (a,b,x 0).  We  will  show  a  figure 
corresponding  to  each  set  of  simulated  data  plotted  with  the  estimated  curve 
for  one  of  the  initial  guesses.  We  will  also  examine  the  standard  errors  of  j3n 
by  uniformly  sampling  varied  amounts  of  data  with  n  =  n0  +  n  1  +  U2  where 
nt  represents  the  amount  of  data  sampled  from  region  Ri}  i  =  0, 1,  2. 
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6.1  Region  Rq 


When  minimizing  the  OLS  functional  with  data  taken  from  region  Rq,  the 
model  with  the  resulting  parameters  closely  approximates  the  data  curve 
within  Rq;  however,  as  seen  in  Figure  3,  it  does  not  come  close  to  the  data 
set  in  other  regions.  We  point  out  that  that  the  graphs  in  Figure  3  correspond 
to  the  first  entries  from  each  section  of  Table  1. 


Figure  3:  Simulated  data  compared  to  the  solution  with  estimated  parame¬ 
ters  using  data  in  the  region  R0  for  the  a)  flat  curve  with  /3°  =  (0.6,  0.2,  0.5) 
b)  moderate  curve  with  /3°  =  (0.8, 0.1, 0.3)  c)  steep  curve  with  /3°  = 
(0.7,  0.2,  0.5). 

When  optimizing  Jn{/3),  we  minimize  the  difference  between  model  solu¬ 
tion  and  data  only  in  the  region  of  interest.  Note  that  the  values  for  J13(/313) 
using  data  from  Rq  are  close  to  zero  in  every  case  even  when  (3 13  is  not  close 
to  /3q.  Because  of  our  previous  analytical  analysis,  we  are  not  surprised  that 
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The  flat  curve  with  (30  =  (0.5,  0.1,  0.1)  where  K=5. 


(3° 

(313 

Jis(/313) 

K 

(0.6, 0.2, 0.5) 

5.8587 

(0.5415,0.3317,0.0991) 

.3766e-5 

1.6328 

(2,0.5, 1) 

89.7757 

(0.6842,1.1442,0.0966) 

.7833e-4 

0.598 

(0.7,0.3,0.001) 

0.398 

(0.6132,0.7294,0.0976) 

.2799e-5 

0.8407 

(5,5,5) 

33.991 

(2.391,10.7157,0.0691) 

0.0058 

0.2231 

The  moderate  curve  with  /30  =  (0.7,0.04,0.1)  where  K=17.5. 


(3° 

Jis((3°) 

(313 

Jis(/313) 

k 

(0.8, 0.1, 0.3) 

3.6193 

(0.7281,0.1552,0.0991) 

.4887e-5 

4.6907 

(2, 0.5, 0.4) 

46.2021 

(0.9510,1.0667,0.0919) 

.395e-a 

0.8915 

(0.3,0.05,1) 

14.9077 

(0.6920,0.0076,0.1003) 

.3658e-6 

91.5778 

(5,5,5) 

33.0287 

(3.1546,10.0916,0.0407) 

0.0239 

0.3126 

The  steep  curve  with  (30  =  (0.8,0.01,0.1)  where  K=80. 


(3° 

$13 

Jis($13) 

k 

(0.7, 0.2, 0.5) 

6.0063 

(0.8339,0.1273,0.0987) 

.1123e-4 

6.5524 

(0.9,0.005,0.2) 

1.7988 

(0.799,0.0064,0.1) 

.6183e-8 

125.7549 

(0.6,0.03,0.9) 

28.6298 

(0.8015,0.015,0.0999) 

.3084e-7 

53.3319 

(5,5,5) 

32.4756 

(3.6576,9.7304,0.0246) 

0.0442 

0.3759 

Table  1:  The  initial  and  optimized  parameters  from  region  R0. 


the  OLS  procedure  does  not  lead  to  accurate  estimates  for  the  values  of  a  or 
b  in  this  region,  and  we  can  see  from  Table  1  that  the  numerical  results  sup¬ 
port  this  previous  analysis.  Moreover,  as  expected,  the  estimation  procedure 
did  return  a  reasonable  estimate  for  the  true  value  of  Xq. 

6.2  Region  R\ 

Note  in  Figure  4  that  by  minimizing  with  data  from  region  Ri,  the  best  fit 
model  provides  a  decent  approximation  in  Ri  as  well  as  in  the  other  regions. 
This  is  what  we  predicted  based  on  the  previous  analysis  regarding  the  xTX 
matrix.  The  graphs  in  the  figure  correspond  to  the  initial  entries  from  each 
section  of  Table  2.  The  solution  is  dynamically  changing  in  R\  with  the 
asymptotes  in  the  individual  curves  lying  ontside  of  this  region.  Hence  it  is 
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possible  to  obtain  reasonable  estimates  for  true  values  /30  using  data  sampled 
from  R\ . 


The  flat  curve  with  p°,  =(  0.6, 0.2, 0.5).  The  moderate  curve  with  p®,  =(  0.8, 0.1 , 0.3). 


Figure  4:  Simulated  data  compared  to  the  solution  with  estimated  pa¬ 
rameters  using  data  from  the  region  Ri  for  the  a)  flat  curve  with  /3°  = 
(0.6,  0.2,  0.5)  b)  moderate  curve  with  (3°  =  (0.8,  0.1, 0.3)  c)  steep  curve  with 
p°  =  (0.7,  0.2,  0.5). 


The  minimized  cost,  J6 i(/361),  is  very  small  for  reasonable  initial  guesses 
as  is  seen  in  Table  2.  When  we  used  the  initial  guess  of  /3°  =  (5.0,  5.0,  5.0),  the 
procedure  returned  a  poor  estimate  of  the  parameter  values.  However,  when 
reasonable  initial  guesses  were  used,  the  values  of  /?o  were  nearly  exactly 
approximated.  This  illustrates  the  local  convergent  properties  of  the  OLS 
procedure. 
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The  flat  curve  with  (3q  =  (0.5,  0.1,  0.1)  where  K=5. 


0° 

Jei(/30) 

(361 

Jei(/361) 

k 

(0.6, 0.2, 0.5) 

68.5101 

(0.5, 0.1, 0.1) 

.1449e-6 

4.7 

(2.0, 0.5,1) 

319.5682 

(0.4999,0.1,0.1) 

.4067e-6 

5 

(0.7,0.3,0.001) 

263.4515 

(0.5, 0.1, 0.1) 

.1757e-6 

4.9999 

(5,5,5) 

204.9234 

(6.4942,2.9899,5.1836) 

121.1291 

2.1721 

The  moderate  curve  with  /3q  =  (0.7,0.04,0.1)  where  K=17.5. 


(3° 

Jei(/30) 

(361 

Je  i(/361) 

k 

(0.8, 0.1, 0.3) 

1.3315e3 

(0.6999,0.04,0.1) 

.1052e-4 

17.4997 

(2.0, 0.5, 0.4) 

3.2091e3 

(0.7,0.04,0.1) 

.1117e-4 

17.4980 

(0.3,0.05,1) 

2.6895e3 

(0.7,0.04,0.1) 

.8611e-6 

17.4988 

(5,5,5) 

5.2903e3 

(7.6425,0.938,5.9823) 

,2173e4 

8.1481 

The  steep  curve  with  (30  =  (0.8,0.01,0.1)  where  K=80. 


(3° 

Jei(/30) 

$61 

Je  i(/361) 

k 

(0.7, 0.2, 0.5) 

8.7989e4 

(0.7999,0.01,0.1) 

.3903e-3 

79.9890 

(0.9,0.005,0.2) 

2.5627e4 

(0.7998,0.01,0.1) 

.3798e-3 

79.99 

(0.6,0.03,0.9) 

4.7167e4 

(0.7999,0.01,0.1) 

.3864e-3 

79.9864 

(5,5,5) 

9.6307e4 

(7.9785,0.2681,4.9371) 

,458e5 

29.7608 

Table  2:  The  initial  and  optimized  parameters  using  data  from  region  R\ . 


6.3  Region  R2 

When  carrying  out  the  OLS  estimation  problem  with  data  selected  form  R2, 
we  observe  (Figure  5,  Table  3)  that  each  model  curve  with  the  estimated 
parameters  is  a  good  approximation  to  the  data  as  the  curves  approach  the 
carrying  capacity  K .  However,  when  we  compare  the  model  with  estimated 
parameters  to  the  data  over  the  entire  region,  we  see  that  it  is  not  in  rea¬ 
sonable  agreement.  In  particular,  the  only  closely  approximated  part  of  the 
data  is  x°°  =  K  =  f . 

Moreover,  when  we  use  data  only  from  region  R-2l  the  estimate  for  each 
parameter  is  poor,  which  is  consistent  with  our  earlier  analytical  consider¬ 
ations.  Recall  that  the  analysis  of  xTX  f°r  sampling  in  R-2  suggests  that  it 
might  be  possible  to  approximate  the  relationship  between  a  and  b  without 
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The  flat  curve  with  p°5  =(  0.6, 0.2, 0.5).  The  moderate  curve  with  p°5  =(  0.8, 0.1 , 0.3). 


The  steep  curve  with  p°5  =(  0.7, 0.2,  0.5). 


Figure  5:  Simulated  data  plotted  versus  the  model  with  estimated  parame¬ 
ters  using  data  from  region  R-2  for  the  a)  flat  curve  with  (3°  =  (0.6,  0.2,  0.5) 
b)  moderate  curve  with  (3°  =  (0.8, 0.1, 0.3)  c)  steep  curve  with  /3°  = 
(0.7,  0.2,  0.5). 


independently  estimating  the  parameters.  Indeed,  Table  3  reveals  that  we 
do  obtain  a  reasonable  approximation  for  K .  Furthermore,  note  that  we  are 
not  able  to  estimate  xq  with  data  from  R-2,  and  as  a  result  the  optimized 
value  Xq  is  near  the  initial  gness  instead  of  the  true  value.  The  algorithm 
cannot  improve  much  on  the  initial  gness  for  Xq  when  using  data  from  R2 
and  hence  leaves  it  essentially  unchanged  from  the  initial  gness.  Similarly,  it 
appears  that  the  optimized  value  a  is  relatively  close  to  the  initial  gness  for 
ao,  whereas  b  is  adjusted  accordingly  to  produce  the  optimized  value  of  K, 
the  only  quantity  to  which  the  data  in  i?2  provides  any  sensitivity. 
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The  flat  curve  with  (30  =  (0.5,  0.1,  0.1)  where  K=5. 


(3° 

J25(/3°) 

/325 

J25(/325) 

k 

(0.6, 0.2, 0.5) 

77.5157 

(0.3539,0.0694,0.4791) 

0.0025 

5.1007 

(2,0.5, 1) 

14.5852 

(1.9241,0.4049,1.0643) 

0.4672 

4.7515 

(0.7,0.3,0.001) 

188.931 

(0.8837,0.1805,0.0011) 

0.0144 

4.8971 

(5,5,5) 

352.3011 

(7.3549,1.5480,5.3033) 

0.4681 

4.7513 

The  moderate  curve  with  /3q  =  (0.7,0.04,0.1)  where  K=17.5. 


(3° 

J25(/3°) 

/325 

J25(/325) 

k 

(0.8, 0.1, 0.3) 

2.151e3 

(1.0057,0.0582,0.3249) 

0.7889 

17.2699 

(2, 0.5, 0.4) 

4.403e3 

(2.7639,0.16,0.4508) 

0.7971 

17.2696 

(0.3,0.05,1) 

3.4282e3 

(0.5051,0.0287,0.9384) 

0.0066 

17.5785 

(5,5,5) 

6.6184e3 

(7.6593,0.4435,5.9774) 

0.8069 

17.2697 

The  steep  curve  with  (30  =  (0.8,0.01,0.1)  where  K=80. 


(3° 

J25(/3°) 

/325 

J25(/325) 

k 

(0.7, 0.2, 0.5) 

1.4142e5 

(1.0855,0.0138,0.597) 

31.6162 

78.7080 

(0.9,0.005,0.2) 

2.5195e5 

(0.7263,0.0091,0.2402) 

0.0308 

80.1117 

(0.6,0.03,0.9) 

8.6528e4 

(0.5574,0.0069,1.7452) 

0.3821 

80.4821 

(5,5,5) 

1.5097e5 

(8.0185,0.1019,6.1419) 

32.1897 

78.7013 

Table  3:  The  initial  and  optimized  parameters  with  data  from  region  R2. 


6.4  Standard  Errors 

When  we  examine  the  standard  errors  for  (3n  as  calculated  in  equation  (9), 
we  obtain  a  measure  of  the  reliability  (i.e.,  uncertainty  associated  with)  the 
estimated  parameter  values.  Specifically,  as  more  points  are  sampled  from 
the  asymptotic  portion  of  the  curve,  the  values  of  the  estimated  parameters 
may  or  may  not  improve  and  the  SE  may  or  may  not  decrease  in  size.  In 
addition  to  using  noise  free  simulated  data  as  in  the  previous  computations, 
we  will  next  introduce  noise  into  the  data  and  examine  the  corresponding 
resulting  standard  errors  for  the  OLS  estimation  procedures. 

As  in  the  case  for  estimation  using  noise  free  data,  by  sampling  data  from 
Rq  alone,  we  are  unable  to  estimate  the  values  for  a  or  b,  and  similarly,  when 
data  is  sampled  from  R2  alone,  we  are  unable  to  estimate  the  values  for  a, 
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b,  or  Xq.  Even  if  the  number  of  points  sampled  from  each  region  is  doubled, 
we  expect  as  before  that  the  estimates  will  not  improve.  However,  if  the 
additional  points  are  instead  sampled  from  a  portion  of  Ri,  we  suspect  that 
the  estimates  for  f3n  might  improve.  In  addition  to  more  accurate  estimates, 
the  standard  errors  may  decrease  in  size  when  Ri  is  included  in  the  sampling 
region. 


The  flat  curve  with  /30  =  (0.5, 0.1,  0.1)  using  /3°  =  (0.6, 0.2,  0.5). 


n0 

n  i 

$n 

Standard  Errors 

25 

0 

(0.5418,0.3345,0.0991) 

(0.011,0.0078,0.0051) 

49 

0 

(0.5413,0.332,0.0991) 

(0.0216,0.0153,0.01) 

25 

24 

(0.5, 0.1, 0.1) 

(7.5111e-5, 1.8985e-5,  5.2494e.-5) 

The  moderate  curve  with  /30  =  (0.7,0.04,0.1)  using  (3°  =  (0.8,  0.1,  0.3). 


no 

ni 

(3n 

Standard  Errors 

25 

0 

(0.7244,0.1414,0.0992) 

(0.0049,0.0011,0.0025) 

49 

0 

(0.7241,0.1401,0.0992) 

(0.0094,0.0020,0.0049) 

25 

24 

(0.6998,0.0397,0.1) 

(2.2059e-4, 1.4424e“5, 1.5696e.-4) 

The  steep  curve  with  /30  =  (0.8,0.01,0.1)  using  /3°  =  (0.7,  0.2,  .5). 


n0 

n\ 

(3n 

Standard  Errors 

25 

0 

(0.8325,0.1238,0.0987) 

(0.0074,0.0012,0.0036) 

49 

0 

(0.8334,0.1280,0.0987) 

(0.0155,0.0026,0.0075) 

25 

24 

(0.8,0.01,0.1) 

(7.0893e-5, 1.0211e-6,  5.7533e.-5) 

Table  4:  The  estimated  parameters  for  (3  =  (a,  b,  x0 )  along  with  standard 
errors  for  the  optimized  f3n. 


6.4.1  Simulated  Data  without  Noise 

We  Erst  sampled  25  data  points  from  the  region  Rq  of  the  data  set  with  no 
noise.  These  points  are  obtained  using  a  uniform  grid  with  the  increment  of 
1/12  over  the  region  [0,2],  Note  that  in  the  corresponding  results  given  in 
Table  4,  af0  is  close  to  the  true  value  of  x0,  whereas  the  estimates  for  a  and 
b  are  not  simultaneously  very  accurate  with  the  inaccuracy  increasing  as  the 
steepness  of  the  data  curve  increases.  This  is  consistent  with  our  results  from 
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Table  1.  In  attempts  to  improve  the  optimized  parameter  values  as  well  as  the 
standard  errors,  we  first  refine  the  grid  size  within  Rq.  Using  the  increment 
of  1/24  over  the  region  [0,2],  we  uniformly  sampled  49  points,  and  observed 
that  /349  is  nearly  equal  to  /325  ,  although  the  standard  errors  doubled  in 
size.  Since  there  is  no  marked  improvement  in  (3n°  by  increasing  the  number 
of  sampled  points  from  region  R0,  we  instead  doubled  the  sampling  region, 
and  now  included  points  sampled  from  R,\ .  Using  the  increment  of  1/12 
uniformly  over  the  region  [0,4],  we  sampled  49  points  from  region  RqU  R\. 
Using  this  data  set  it  is  seen  in  Table  4  that  /349  is  a  close  approximation  to 
the  true  value  of  (30,  and  the  standard  errors  are  reduced  by  several  orders 
of  magnitude. 


The  flat  curve  with  /30  =  (0.5, 0.1,  0.1)  using  (3°  =  (0.6, 0.2,  0.5). 


n  i 

n2 

(3n 

Standard  Errors 

0 

25 

(0.3539,0.0694,0.4791) 

(0.0702,0.0182,0.1684) 

0 

49 

(0.3190,0.0621,0.6709) 

(0.1668,0.0435,0.4931) 

24 

25 

(0.5, 0.1, 0.1) 

(7.6421e-4, 1.9291e-4,  5.3446e.-4) 

The  moderate  curve  with  (30  =  (0.7,0.04,0.1)  using  (3°  =  (0.8,  0.1,  0.3). 


ni 

ri2 

(3n 

Standard  Errors 

0 

25 

(1.0057,0.0582,0.3249) 

(1.0794,0.0663,1.3714) 

0 

49 

(1.0058,0.0582,0.3249) 

(2.1739,0.1335,2.7618) 

24 

25 

(0.6996,0.04,0.1003) 

(5.0686e-3,  3.3366e-4,  3.6103e-3) 

The  steep  curve  with  (30  =  (0.8,0.01,0.1)  using  f3°  =  (0.7,  0.2,  0.5). 


ni 

r>2 

$n 

Standard  Errors 

0 

25 

(1.0855,0.0138,0.5970) 

(1.6669,0.0224,4.4595) 

0 

49 

(1.0855,0.0138,0.5970) 

(3.3488,0.045,8.9585) 

24 

25 

(0.7996,0.01,0.1002) 

(1.4895e-2,  2.1398e-4, 1.2125e.-2) 

Table  5:  The  estimated  parameters  for  (3  =  (a,  b ,  x0)  along  with  standard 
errors  for  the  optimized  (3n. 


We  turn  to  similar  computations  for  data  from  i?2  and  i?2  U  R\.  We 
sampled  25  data  points  from  region  i?2,  where  these  points  are  obtained 
using  a  uniform  grid  with  the  increment  of  1/6  over  the  region  [12, 16].  Note 
in  Table  5  that  the  values  for  K  =  a/b  are  reasonably  close  to  the  true 
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values  of  K  =  5, 17.5,  and  80,  respectively  for  the  three  data  curves.  The 
corresponding  estimates  for  a,  b  and  x$  are  not  accurate  for  any  of  the  curves. 
This  is  consistent  with  our  findings  reported  in  Table  3.  In  an  initial  attempt 
to  improve  the  optimized  values  as  well  as  the  standard  errors,  we  refined  the 
sampling  grid  within  R2.  Using  the  increment  of  1/12  over  the  region  [12, 16], 
we  uniformly  sampled  49  points.  We  observe  that  /349  is  nearly  equivalent  to 
/325,  and  as  before  with  data  in  R0,  the  standard  errors  double  in  size.  Since 
there  is  no  improvement  in  (3n 2  by  increasing  the  number  of  sampled  points 
from  region  R2l  we  next  doubled  the  sampling  region  and  included  points 
sampled  from  R,\ .  Thus,  using  the  increment  of  1/6  over  the  region  [8,16], 
we  sampled  49  points  from  the  region  Ri  U  R2.  Using  this  data  set,  /349  is 
a  reasonable  approximation  to  the  true  value  of  (3 o,  and  the  standard  errors 
are  significantly  reduced. 

6.4.2  Simulated  Data  with  Noise 

We  considered  two  different  sets  of  simulated  data  with  noise  corresponding 
to  the  moderate  curve,  i.e.,  (3q  =  (0.7,0.04,0.1).  One  set  corresponds  to  the 
data  in  R0  which  is  perturbed  by  adding  Gaussian  noise  ej  l'~'  A/”(0,  cr0)  for 
j  —  1,2, ,  n,  with  (T0  =  0.005.  The  other  data  set  corresponds  to  data  in  R2 
with  similar  added  noise  except  with  <To  =  0.5.  These  sigma  values  represent 
an  added  noise  level  of  approximately  ten  percent  at  the  lowest  bound  in 
the  observed  regions.  Figure  6  depicts  the  data  set  with  the  highest  level  of 
noise. 


Figure  6:  Noisy  data  with  <r0  =  0.5  compared  to  the  exact  solution  with 
Po  =  (0.7,0.04,0.1). 
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The  moderate  curve  with  (30  =  (0.7,0.04,0.1)  using  (3°  =  (0.8,  0.1,  0.3). 


no 

n\ 

{3n 

Standard  Errors 

25 

0 

(0.7244,0.1455,0.0988) 

(0.052,0.0116,0.0269) 

49 

0 

(0.7245,0.1452,0.0988) 

(0.1060,0.0236,0.0548) 

25 

24 

(0.7086,0.0517,0.0988) 

(0.0497,0.0041,0.033) 

49 

48 

(0.703,0.0435,0.0994) 

(0.091,0.0065,0.0631) 

0 

25 

(0.7058,0.0491,0.0994) 

(0.0133,0.0011,0.009) 

0 

49 

(0.6958,0.0362,0.1008) 

(0.0252,0.0015,0.0184) 

Table  6:  The  standard  errors  for  the  optimized  /3n  with  <jq  =  0.005  using 
data  sampled  from  the  interval  [0,4], 


We  uniformly  sampled  data  with  a  step  size  of  1/12  from  the  intervals 
[0,2],  [0,4],  and  [2,4],  Then  the  sampling  frequency  is  doubled  over  the  same 
set  of  intervals  to  determine  if  refining  the  grid  will  result  in  better  estimates 
with  reasonable  standard  errors.  By  examining  the  [0,2]  interval,  we  deter¬ 
mine  that  the  j3 25  does  not  accurately  estimate  /30  and  the  standard  errors 
double  as  a  result  of  the  grid  refinement.  As  seen  in  Table  6,  the  estimates 
and  the  standard  errors  improve  somewhat  when  sampling  from  [0,4],  How¬ 
ever,  doubling  the  sampling  frequency  increases  the  standard  errors  without 
significant  improvement  to  the  estimates.  When  sampling  from  the  interval 
[2,4],  we  observed  improved  parameter  estimates  with  reduced  standard  er¬ 
rors.  Thus  we  see  that  including  the  Ro  region  within  the  data  set  results  in 
worse  estimates  and  larger  standard  errors  than  sampling  from  R\  alone. 


The  moderate  curve  with  /30  =  (0.7,0.04,0.1)  using  (3°  =  (0.8,  0.1,  0.3). 


n  i 

Interval 

(3n 

Standard  Errors 

25 

[2,4] 

(0.7058,0.0491,0.0994) 

(0.0133,0.0011,0.009) 

49 

[2,4] 

(0.6958,0.0362,0.1008) 

(0.0252,0.0015,0.0184) 

49 

P.6] 

(0.6981,0.0394,0.1006) 

(0.0355,0.0023,0.0255) 

73 

[2.8] 

(0.6999,0.04,0.1001) 

(0.0794,0.0052,0.0564) 

Table  7:  The  standard  errors  for  the  optimized  f3n  with  cr0  =  0.005  using 
data  sampled  from  various  intervals  within  R\ . 


Instead  of  only  increasing  the  sampling  frequency  over  a  fixed  interval, 
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we  also  considered  increasing  the  sampling  region  to  determine  if  there  is  an 
improvement  in  parameter  estimation.  Based  on  our  hirelings  summarized  in 
Table  6,  we  focused  on  sampling  data  points  from  region  R\  alone.  We  have 
observed  that  when  a  given  parameter  estimate  is  already  close  to  the  true 
value,  the  standard  error  corresponding  to  that  parameter  becomes  discern- 
ably  worse  as  the  sampling  region  increases.  In  this  case,  even  though  n  is 
increased  in  a  region  of  high  sensitivity  to  all  parameters,  the  OLS  proce¬ 
dure  is  not  improved  with  additional  data  since  the  additional  data  contains 
only  information  already  obtained  at  the  lower  sampling  level,  i.e.,  the  ill- 
conditioning  of  xTX  is  increasing.  These  results  are  displayed  in  Table  7. 


The  moderate  curve  with  /30  =  (0.7,0.04,0.1)  using  /3°  =  (0.8,  0.1,  0.3). 


n  i 

r>2 

(3n 

Standard  Errors 

0 

25 

(1.0058,0.0582,0.3249) 

(2.7953,0.1716,3.5515) 

0 

49 

(1.0057,0.0582,0.3249) 

(6.3606,0.3906,8.0808) 

24 

25 

(0.5286,0.0297,0.4397) 

(3.7015,0.2516,10.375) 

48 

49 

(0.673,0.0384,0.123) 

(10.2462,0.679,8.8898) 

25 

0 

(0.5175,0.0281,0.4307) 

(1.0073,0.0671,2.8477) 

49 

0 

(0.4713,0.0249,0.5921) 

(2.1167,0.1414,8.0601) 

Table  8:  The  standard  errors  for  the  optimized  /3n  with  a0  =  0.5  using  data 
sampled  from  the  interval  [8, 16]. 


We  uniformly  sampled  data  with  a  step  size  of  1/6  from  the  intervals 
[12,16],  [8,16],  and  [8,12]  in  Table  8.  By  doubling  the  sampling  frequency  over 
the  same  set  of  intervals,  we  again  questioned  if  refining  the  grid  will  result 
in  better  estimates  with  reasonable  standard  errors.  On  the  [12,16]  interval, 
we  determined  that  the  OLS  procedure  does  not  return  a  good  estimate  for 
/30.  When  the  sampling  frequency  is  doubled,  the  poor  parameter  estimate 
capability  remains  and  the  standard  errors  double.  We  included  data  points 
from  the  upper  region  of  R\  by  sampling  from  the  interval  [8,16]  and  there 
is  slight  improvement  in  the  parameter  estimates  and  the  standard  errors. 
However,  when  we  restrict  the  sampling  region  to  [8,12]  the  standard  errors 
improve  significantly.  Therefore  including  R2  in  the  data  set  increases  the 
standard  error  with  minimal  improvement  to  the  parameter  estimates. 

In  Table  9  we  again  increased  the  sampling  region  instead  of  primarily 
increasing  the  sampling  frequency  over  a  fixed  interval  to  determine  if  there 
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The  moderate  curve  with  /30  =  (0.7,0.04,0.1)  using  (3°  =  (0.8,  0.1,  0.3). 


n± 

Interval 

f3n 

Standard  Errors 

25 

[8,12] 

(0.5175,0.0281,0.4307) 

(1.0073,0.0671,2.8477) 

49 

[8,12] 

(0.4713,0.0249,0.5921) 

(2.1167,0.1414,8.0601) 

49 

[4,12] 

(0.6720,0.0383,0.1254) 

(5.1571,0.3409,4.5525) 

Table  9:  The  standard  errors  for  the  optimized  /3n  with  <7o  =  0.5  using  data 
sampled  from  various  intervals  within  R\ . 


is  an  improvement  in  parameter  estimates.  In  light  of  the  results  from  Table 
8,  we  focused  on  sampling  data  points  from  region  Ri  alone.  As  the  sampling 
region  expands  to  include  more  of  the  curve  in  R,\ ,  a  better  estimate  for  the 
parameters  /30  is  obtained.  While  we  are  able  to  improve  our  parameter 
estimates,  the  corresponding  standard  errors  are  again  increased  (compare 
with  Table  7). 

7  Concluding  remarks 

In  summary  we  emphasize  several  points  that  are  readily  illustrated  by  our 
combination  of  analytical  and  computational  analysis  in  this  note.  In  inverse 
problems,  parameter  sensitivity  is  of  fundamental  importance  in  several  re¬ 
gards: 

(i)  Lack  of  sensitivity  can  produce  the  inability  to  even  estimate  correspond¬ 

ing  parameters; 

(ii)  As  illustrated  in  Table  7,  even  when  parameter  sensitivity  is  adequate, 

increased  sampling  (i.e.,  larger  n)  does  not  necessarily  significantly 
improve  the  estimates  nor  does  it  necessarily  improve  at  all  the  re¬ 
liability  (i.e.,  the  SE  and  Cl)  of  the  estimates  obtained.  In  particu¬ 
lar,  ill-conditioning  of  the  matrix  xTX  can  increase  as  data  points  are 
added  without  adding  new  information  (i.e.,  adding  a  row  to  x  that 
is  almost  linearly  dependent  on  previous  rows).  This  provides  an  in¬ 
creasing  (xTx)jJ1  and  hence  an  increasing  SEj  with  increasing  n  even 
if  a2  =  yA_( RSS )  is  decreasing.  This  is  especially  likely  with  increased 
sampling  near  steady-states  or  equilibria  of  a  dynamical  system  where 
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often  little  new  information  about  dynamics  is  provided  with  additional 
sampling. 

In  practice  one,  of  course,  usually  does  not  know  the  exact  solution  and 
may  not  know  all  the  qualitative  properties  of  the  dynamics.  But  the  analysis 
and  computations  in  this  note  in  the  context  of  asymptotic  statistical  esti¬ 
mates  for  covariance  matrices  and  standard  errors  suggest  that  one  might 
profitably  use  the  sensitivities  in  a  problem  to  design  experiments  for  addi¬ 
tional  data  collection. 
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